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I. REAL PARTY IN INTEREST 

The subject application is owned by Microsoft Corporation of Redmond, Washington. 
H. RELATED APPEALS AND INTERFERENCES 

Upon information and belief, appellants do not have any knowledge of related appeals or 
interferences that may directly affect or have a bearing on the decision of the Board of Appeals 
and Interferences (hereinafter "the Board") in pending appeal. 

m. STATUS OF THE CLAIMS 

On December 31, 1998, appellants filed the pending patent application along with 
Claims 1-44. On January 16, 2002, the Examiner issued a first Office Action rejecting 
Claims 1-44. On April 15, 2002, appellants filed an amendment and response in which Claims 1, 
13, 16, 18, 24, 36, 38, and 40 were amended. 

On July 8, 2002, the Examiner issued a second Office Action, finally rejecting 
Claims 1-44. On October 28, 2002, appellants filed an amendment and response in which 
Claims 1, 6, 16, 18, 24, 29, 38, and 40 were amended; and Claims 10 and 33 were canceled. 

On November 6, 2002, the Examiner issued an Advisory Action, 1 rejecting Claims 1-44 
and did not enter the proposed amendments. On November 7, 2002, appellants filed a Request 
for Continuing Examination, submitting by reference the amendment and response filed on 
October 28, 2001. 

On January 30, 2003, the Examiner issued a third Office Action rejecting Claims 1-9, 
11-32, and 34-44. On April 30, 2003, appellants filed an amendment and response in which 
Claims 3, 6-9, 11, 12, 14, 18, 29-32, 34, 35, and 40 were amended; and Claims 13 and 36 were 
canceled. 



1 The Advisory Action mailed on November 6, 2002 was a "draft" Advisory Action. The Examiner mailed 
a final version of the same Advisory Action on November 12, 2002. 
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On July 15, 2003, the Examiner issued a fourth Office Action, finally rejecting 
Claims 1-9, 11, 12, 14-32, 34, 35, and 37-44. On September 15, 2003, appellants filed a 
Response to the Final Office Action. 

On October 20, 2003, the Examiner issued an Advisory Action still rejecting Claims 1-9, 

11, 12, 14-32, 34, 35 and 37-44. On November 12, 2003, appellants filed a Request for 
Continuing Examination, submitting an amendment and response in which Claims 1, 6-9, 16, 18, 
24, 29-32, 38, and 40 were amended. 

On February 2, 2004, the Examiner issued a fifth Office Action rejecting Claims 1-9, 11, 

12, 14-32, 34, 35, and 37-44. This appeal followed on April 20, 2004. Appellants request the 
Board to reverse the rejections of Claims 1-9, 11, 12, 14-32, 34, 35, and 37-44. The claims on 
appeal are set forth in Appendix A. 

IV. STATUS OF AMENDMENTS 

On April 15, 2002, appellants filed an amendment and response in which Claims 1, 13, 
16, 18, 24, 36, 38, and 40 were amended. On November 7, 2002, appellants filed a Request for 
Continuing Examination, submitting by reference the amendment and response filed on 
October 28, 2001, in which Claims 1, 6, 16, 18, 24, 29, 38, and 40 were amended; and Claims 10 
and 33 were canceled. On April 30, 2003, appellants filed an amendment and response in which 
Claims 3, 6-9, 11, 12, 14, 18, 29-32, 34, 35, and 40 were amended; and Claims 13 and 36 were 
canceled. On November 12, 2003, appellants filed a Request for Continuing Examination, 
submitting an amendment and response in which Claims 1, 6-9, 16, 18, 24, 29-32, 38, and 40 
were amended. All of the above amendments have been entered. 

V. SUMMARY OF THE INVENTION 

Prior to discussing appellants' invention, a brief background of appellants' invention is set 
forth to help the Board better appreciate appellants 1 invention discussed thereafter. The 
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following background and the discussions of the disclosed embodiments of appellants' invention 
are not provided to define the scope or interpretation of any of the claims of this application. 
A. Background of the Invention 

Microsoft CLIP GALLERY is a multimedia application program produced by Microsoft 
Corporation, Redmond, Washington, for previewing, and inserting into documents, multimedia, 
such as clip art, pictures, sounds, video clips, and animation art (collectively referred to as 
"clips"). Information about clips is stored in a catalogue. The catalogue contains a small 
graphical representation, known as a thumbnail, of the clip, as well as information about the clip, 
such as the artistic style of the clip, keywords that describe the clip (or some aspect of the clip), 
categories to which the clip belongs, and the location of the clip. Clips in a CLIP GALLERY 
catalogue are organized according to type of clip (e.g., sound, video, picture, etc.). A user can 
also use the CLIP GALLERY program to organize clips by category or keyword. The CLIP 
GALLERY program can be invoked stand-alone or by another application program, such as 
Microsoft WORD, also produced by Microsoft Corporation, Redmond, Washington. Within a 
category (or the entire CLIP GALLERY catalogue), a user may search by manually entering 
keywords. Application, page 1, line 8 to page 1, line 21. 

In the past, it has been difficult to perform successive searches through a large volume of 
clips included in the catalogues of a CLIP GALLERY program and other similar programs. As a 
result of this, users often do not succeed in finding the image they need, even if that image is 
available in the gallery. This result is partially true because many users of such programs are not 
skilled at formulating complex queries. In many instances, users, even skilled users, do not know 
what search term(s) to apply because graphics and sounds are difficult to describe in words. As a 
result, a suitable search tool should be able to accommodate a user who does not know exactly 
what he or she is looking for, but will "know it when I see it." Additionally, the search tool 
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should assist a user in finding clips that match an example clip in visual style so that a document 
can be produced that contains graphics that are consistent in appearance. The present invention 
is directed to providing such a search tool. Application, page 1 , line 22 to page 2, line 10. 
B. Summary of the Invention 

The invention is directed to a method, apparatus, and computer-readable medium for 
searching media clip databases associated with a media application program wherein the media 
clip database includes find similar clips indicia and keywords. Application, page 2, lines 12-17. 
The find similar clips indicia include hidden criteria that identifies and/or groups media clips 
based on human judgment regarding the content of the media clips, such as artistic style, color, 
or shape. Application, page 10, lines 23-28; page 2, lines 24-27. The find similar clips indicia 
and the keywords describe associated media clips stored in the media clip database. 

The invention provides a method that generally comprises, in response to a user selecting 
a media clip, retrieving information, including find similar clips indicia and keywords, associated 
with the selected media clip from the media clip database. Application, page 8, lines 21-34. 
This form of the method further comprises simultaneously presenting to the user for selection by 
the user (i) the keywords associated with the selected media clip and (ii) the find similar clips 
indicia associated with the selected media clip. Application, page 9, line 24 to page 10, line 6. 
As noted above, the find similar clips indicia includes hidden criteria that identifies and/or 
groups media clips based on human judgment regarding the content of the media clips. The 
method also comprises, in response to the user creating search criteria by selecting one or more 
of the keywords associated with the selected media clip and/or the find similar clips indicia 
associated with the selected media clip, retrieving all media clips in the media clip database that 
match the search criteria created by the user. Application, page 10, lines 17-19; page 11, 
lines 4-22. 
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The invention also provides a user interface for a visual thesaurus for a media clip 
database associated with a multimedia application program. Application, page 7, line 3 1 to 
page9 5 line 21. The media clip database contains information, including find similar clips 
indicia and keywords, associated with the media clips included in the media clip database. The 
find similar clips indicia include hidden criteria that identifies and/or groups media clips based 
on human judgment regarding the content of the media clips. Directly in response to the user 
selecting a media clip from the media clip database, the method provides for displaying to the 
user an option for finding similar media clips that have an associated find similar clips indicia 
and/or a keyword that matches the find similar clips indicia human judgment and/or a keyword 
associated with the selected clip. 

The invention also provides a computer-readable medium having computer-executable 
instructions for performing the foregoing methods. Application, page 6, lines 11-14. 

The invention further provides an apparatus for searching a plurality of media clips 
comprising a processing unit and a storage medium coupled to the processing unit. Application, 
page 5, lines 30-34. The storage medium stores program code implemented by the processing 
unit for (i) providing an interface for a user to select a media clip from a media clip database 
associated with a multimedia application program, wherein the media clip database contains 
information, including find similar clips indicia and keywords associated with media clips in the 
media clip database, the find similar clips indicia including hidden criteria that identifies and/or 
groups media clips based on human judgment regarding the content of the media clips; 
(ii) providing an interface for the user to select search criteria based on find similar clips indicia 
and/or a keyword associated with the selected media clips; and (iii) in response to the user 
selecting a media clip and the search criteria, retrieving all media clips in the media clip database 
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that have associated find similar clips indicia and/or a keyword that matches the selected search 
criteria for the selected media clip. Application, page 7, lines 22-30. 

The invention also provides an apparatus for providing a user interface for a visual 
thesaurus for a media clip database associated with a multimedia application program wherein 
the media clip database contains information, including find similar clips indicia and keywords 
associated with media clips in the media clip database. The find similar clips indicia includes 
hidden criteria that identifies and/or groups media clips based on human judgment regarding the 
content of the media clips. The apparatus comprises a processing unit and a storage medium 
coupled to the processing unit. Application, page 5, lines 30-34. The storage medium stores 
program code implemented by the processing unit for displaying to a user an option for finding 
similar media clips that have associated find similar clips indicia hidden criteria and/or a 
keyword that matches an associated keyword for a selected media clip directly in response to the 
user selecting the media clip. Application, page 7, lines 22-30. 

In summary, the method, apparatus, and computer-readable medium incorporating the 
invention employ two techniques that can be used separately or together to select media clips 
from a media clip database keywords associated with a media clip being viewed and/or a find 
similar clips indicia that includes hidden criteria (i.e., criteria not observable by a user and not 
changeable by a user) that identifies and/or groups media clips based on human judgment 
regarding the content of the media clips. As noted above, examples of find similar clips indicia 
having hidden criteria are artistic style, color, and shape. These find similar clips indicia have 
hidden criteria that, as noted above, cannot be modified by a user. 
C. Explanation of the Invention Defined in the Claims 

Regarding the claims, independent Claim 1 is directed to a method of searching a media 
clip database associated with a multimedia application program. The media clip database 
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contains information, including keywords and find similar clips indicia associated with media 
clips included in the media clip database. The find similar clips indicia include hidden criteria 
that identifies and/or groups media clips based on human judgment regarding the content of the 
media clip. The method includes, in response to a user selecting a media clip, retrieving 
information, including find similar clips indicia and keywords, associated with a selected media 
clip from a media clip database. The method further includes simultaneously presenting to the 
user for selection by the user (i) the keywords associated with the selected media clip and (ii) the 
find similar clips indicia associated with the selected media clip. Additionally, the method 
includes, in response to the user creating search criteria by selecting one or more of the keywords 
and/or the find similar clips indicia associated with the selected media clip, retrieving all media 
clips in the media clip database that match the search criteria created by the user. 

Claims 2-9, 11, 12, 14, and 15 are dependent from independent Claim 1 and are directed 
to further limitations of the method described above. Claim 2 depends from Claim 1 and recites 
further comprising presenting a user with all of the retrieved clips. Claim 3 depends from 
Claim 2, and recites further comprising, in response to a user selecting one of the retrieved clips 
and the user creating a search criteria for the newly selected clip, retrieving all clips in the clip 
database that match the search criteria for the newly selected media clip. Claim 4 depends from 
Claim 1 and recites further comprising presenting a plurality of media clips available for 
selection to a user. Claim 5 depends from Claim 4 and recites wherein the media clips are based 
on a media type. Claim 6 depends from Claim 1 and recites wherein the human judgment is 
based on artistic style. Claim 7 depends from Claim 1 and recites wherein the human judgment 
is based on color. Claim 8 depends from Claim 1 and recites wherein the human judgment is 
based on shape. Claim 9 depends from Claim 1 and recites wherein the human judgment is 
based on both color and shape. Claim 1 1 depends from Claim 1 and recites wherein the 
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keywords are a list of keywords. Claim 12 depends from Claim 1 and recites wherein the 
keywords designate the format of the clip. Claim 14 depends from Claim 1 and recites wherein 
the retrieving all clips in the clip database that match the search criteria created by the user 
applies only to clips having a media type that matches the media type of the selected media clip. 
Claim 15 recites a computer-readable medium having computer-executable instructions for 
performing the method recited in any one of Claims 1-9, 1 1, 12, or 14. 

Independent Claim 16 is directed to a method for providing a user interface for a visual 
thesaurus for a media clip database associated with a multimedia application program. The 
media clip database contains information, including find similar clips indicia and keywords 
associated with media clips included in the media clip database. The find similar clips indicia 
include hidden criteria that identify and/or group media clips based on human judgment 
regarding the content of the media clip. The method includes, directly in response to a user 
selecting a media clip from the media clip database, displaying to the user an option for finding 
similar media clips that have an associated find similar clips indicia and/or a keyword that 
matches the find similar clips indicia, human judgment, and/or a keyword associated with the 
selected clip. 

Claims 17-23 depend from independent Claim 16, and are directed to further limitations 
of the method described above. Claim 17 depends from Claim 16 and recites wherein the user 
interface comprises (a) a graphical indication of the selected clip and (b) a button for the option 
for finding similar clips to the selected clip. Claim 18 depends from Claim 16 and recites further 
comprising, in response to the user selecting the option for finding similar clips, displaying 
means for the user to select search criteria for finding similar clips, wherein the search criteria 
includes a keyword that matches the find similar clips indicia, human judgment, and/or a 
keyword associated with the selected clip. Claim 19 depends from Claim 18 and recites wherein 
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means for the user to select search criteria for finding similar clips is a fly-out window. Claim 20 
depends from Claim 16 and recites further comprising displaying to the user an option for 
inserting the selected clip into a document. Claim 21 depends from Claim 16 and recites further 
comprising displaying to the user an option for previewing the selected clip. Claim 22 depends 
from Claim 1 6 and recites further comprising displaying to the user an option to add the clip to 
one or more categories. Claim 23 recites a computer-readable medium having computer- 
executable instructions for performing the method recited in any one of Claims 16-22. 

Independent Claim 24 is directed to an apparatus for searching a plurality of media clips. 
The apparatus includes a processing unit, and storage medium coupled to the processing unit. 
Storage medium storing program code is implemented by the processing unit for (1) providing an 
interface for a user to select a media clip from a media clip database associated with the 
multimedia application program, wherein the media clip database contains information, including 
find similar clips indicia and keywords associated with media clips in the media clip database, 
the find similar clips indicia includes hidden criteria that identifies and/or groups media clips 
based on human judgment regarding the contents of the media clip; (2) providing an interface 
for the user to select search criteria based on find similar clips indicia and/or the keyword 
associated with the selected media clip; and (3) in response to the user selecting the media clip 
and the search criteria, retrieving all media clips in the media clip database that have associated 
find similar clips indicia and/or a keyword that matches the selected search criteria for the 
selected media clip. 

Claims 25-32, 34, 35, and 37 depend from Claim 24, and are directed to further 
limitations of the apparatus described above. Claim 25 depends from Claim 24, and recites 
wherein the program code further presents a user with all of the retrieved clips. Claim 26 
depends from Claim 25, and recites wherein the program code further retrieves all clips in the 
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clip database that match a search criteria for a newly selected media clip, in response to a user 
selecting one of the retrieved clips and the user selecting the search criteria for the newly selected 
clip. Claim 27 depends from Claim 24, and recites wherein the program code further presents a 
plurality of media clips available for selection to a user. Claim 28 depends from Claim 27, and 
recites wherein the media clips are based on a media type. Claim 29 depends from Claim 24 and 
recites wherein the human judgment is based on artistic style. Claim 30 depends from Claim 24 
and recites wherein the human judgment is based on color. Claim 3 1 depends from Claim 24, 
and recites wherein the human judgment is based on shape. Claim 32 depends from Claim 24 
and recites wherein the human judgment is based on color and shape. Claim 34 depends from 
Claim 24, and recites wherein the keywords are a list of keywords. Claim 35 depends from 
Claim 24 and recites wherein the keywords designate the format of the clip. Claim 37 depends 
from Claim 24, and recites wherein retrieving all clips in the clip database that match the search 
criteria for the selected media clip applies only to clips having a media type that matches the 
media type of the selected media type. 

Independent Claim 38 is directed to an apparatus for providing a user interface for a 
visual thesaurus for a media clip database associated with a multimedia application program. 
The media clip database contains information, including find similar clips indicia and keywords 
associated with media clips in the media clip database, the find similar clips indicia including 
hidden criteria that identifies and/or groups media clips based on human judgment regarding the 
content of the media clip. The apparatus includes a processing unit, and a storage medium 
coupled to the processing unit. The storage medium storing program code is implemented by the 
processing unit for displaying to a user an option for finding similar media clips that have 
associated find similar clips indicia hidden criteria, and/or a keyword that matches an associated 
keyword for a selected media clip, directly in response to a user selecting a media clip. 
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Claims 39-44 depend from Claim 38, and are directed to further limitations of the 
apparatus described above. Claim 39 depends from Claim 38, and recites wherein the user 
interface includes the graphical indication of the selected clip and a button for the option for 
finding similar clips to the selected clip. Claim 40 depends from Claim 38 and recites wherein 
the program code displays means for a user to select search criteria that is based on find similar 
clips indicia and/or a keyword associated with the selected clip for finding similar clips, in 
response to the user selecting the option for finding similar clips. Claim 41 depends from 
Claim 40 and recites wherein means for the user to select search criteria for finding similar clips 
is a fly-out window. Claim 42 depends from Claim 38, and recites wherein the program code 
further displays to the user an option for inserting the selected clip into a document. Claim 43 
depends from Claim 38, and recites wherein the program code further displays to the user an 
option for previewing the selected clip. Claim 44 depends from Claim 38 and recites wherein the 
program code further displays to the user an option to add the clip to one or more categories. 

VI. ISSUES PRESENTED FOR REVIEW 

The last Office Action in this patent application, dated February 2, 2004 (hereinafter 
"Office Action"), rejected Claims 1-9, 11-12, 14-32, 34-35, and 37-44 under 35 U.S.C. § 103(a) 
as unpatentable over U.S. Patent No. 5,493,677, issued to Balogh et al. ("Balogh"), in view of 
U.S. Patent No. 5,696,964, issued to Cox et al. ("Cox"). Appellants disagree. Thus, the issues 
for review are as follows: 

(a) Whether independent Claim 1 is unpatentable under 35 U.S.C. § 103(a) over Balogh 
in view of Cox; 

(b) Whether dependent Claims 2-15 are unpatentable under 35 U.S.C. § 103(a) over 
Balogh in view of Cox; 
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(c) Whether independent Claim 16 is unpatentable under 35 U.S. C § 103(a) over Balogh 
in view of Cox; 

(d) Whether dependent Claims 17-23 are unpatentable under 35 U.S.C. § 103(a) over 
Balogh in view of Cox; 

(e) Whether independent Claim 24 is unpatentable under 35 U.S.C. § 103(a) over Balogh 
in view of Cox; 

(f) Whether dependent Claims 25-32, 34, 35, and 37 are unpatentable under 35 U.S.C. 
§ 103(a) over Balogh in view of Cox; 

(g) Whether independent Claim 38 is unpatentable under 35 U.S.C. § 103(a) over Balogh 
in view of Cox; and 

(h) Whether dependent Claims 39-44 are unpatentable under 35 U.S.C. § 103(a) over 
Balogh in view of Cox. 

As will be discussed below, the Examiner has failed to establish a prima facie case of 
obviousness. To establish prima facie obviousness in a claimed invention, all the claim 
limitations must be taught or suggested by the prior art. M.P.E.P. § 2143. All of the independent 
claims in this application (1, 16, 24, and 38) recite that the find similar clips indicia includes 
hidden criteria that identifies and/or groups media clips based on human judgment regarding the 
content of the media clips. This subject matter is not taught or even remotely suggested by either 
of the references cited and applied in the final Office Action — Balogh et al. and Cox et al. — 
either alone or in combination. For a better appreciation of the arguments below, appellants 
summarize each cited and applied reference. 
A. Summary of Balogh et al. 

Balogh, referring to FIGURE 3, generally discloses retrieving images using a natural 
language query. Digitized images are associated with English language captions and other data, 
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collectively known as the metadata 262 associated with the images 250. Col. 3, lines 17-43; 
Col. 8 3 lines 64-67. A natural language processing database removes ambiguities from the 
metadata, and the images in the metadata are stored in databases. Col. 3, lines 44-48. A user 
forms a search query, and natural language processing is used to determine matches between the 
query and the stored metadata. Col. 11, lines 60-67. Captions and corresponding images that 
match queries are returned to the user. Col. 14, lines 3-8. Once some initial matching captions 
are retrieved for the user and reviewed along with their corresponding images, further searches 
may be made by the user dragging the image (and the data) into a new query. Col. 14, 
lines 40-59. Images corresponding to the matches are then viewed, and desired images are 
selected for licensing. The license terms for selected images are displayed and a subset of the 
selected images is ordered as desired by the user. In summary, Balogh discloses retrieving 
images using a natural language interface and English language metadata associated with the 
image. 

Balogh does not disclose retrieving information, including find similar clips indicia and 
keywords, associated with said selected media clip from the database, in response to a user 
selecting a media clip. Balogh also does not disclose displaying keywords associated with a 
selected multimedia clip for selection by a user for subsequent searching. Further, Balogh does 
not disclose find similar clips indicia including hidden criteria that identifies and/or groups 
media clips based on human judgment regarding the content of the media clip. 
B. Summary of Cox et al. 

Cox, referring to FIGURE 10, is generally directed toward a multimedia database 
retrieval system that maintains a "posterior probability distribution" that each item in the 
database is the target of a search. Col. 4, lines 42-64. Cox discloses a queryless multimedia 
database search method, which incorporates a Bayesian interface engine that refines its answer 
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with each user response. Col. 2, lines 18-34. Referring to FIGURE 2, the set of user responses 
includes a set of images (4 as shown) and user actions ("GO," "FOUND," and "ABORT" 
buttons), and is defined by a relatively simple user interface. The system maintains a posterior 
probability distribution that each image in a multimedia database is the target of the search. This 
distribution is used to select the next images to display to the user and solicits selections from the 
user pertaining to images displayed. The user's selections are then incorporated into the posterior 
distribution via a probabilistic user model. Col. 4, line 42 to Col. 5, line 4; FIGURE 10. In 
summary, Cox discloses a queryless multimedia database retrieval method and system that 
maintain a posterior probability distribution for use in selecting the next images to display to the 
user. Cox discloses an entirely software driven selection system. No human judgment is 
involved in the criteria used to identify clips based on user selection. Col. 6, lines 1-23 
(TABLE 1). 

VII. GROUPING OF CLAIMS 
Claims 1-9, 11, 12, and 14-15 stand or fall together; Claims 16-23 stand or fall together; 
Claims 24-32, 34, 35, and 37 stand or fall together; and Claims 38-44 stand or fall together. The 
reasons why the four groups of claims are believed to be separately patentable are explained 
above and are further expounded below in the argument. 

VIII. ARGUMENT 

As discussed in greater detail below, the claims of the present application are clearly 
patentably distinguishable over the teachings of the above-cited references. The present 
invention is directed to a method, apparatus, and computer-readable medium for searching media 
clip databases associated with a media application program wherein the media clip database 
includes find similar clips indicia and keywords. The find similar clips indicia include hidden 
criteria that identifies and/or groups media clips based on human judgment regarding the content 
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of the media clips, such as artistic style, color, or shape. The find similar clips indicia and the 
keywords describe associated media clips stored in the media clip database. 

As noted above, the Office Action rejected Claims 1-9, 11, 12, 14-32, 34, 35, and 37-44 
Under 35 U.S.C. § 103(a) as being unpatentable over Balogh in view of Cox. As also noted 
above, appellants disagree. The cited references simply fail to teach or suggest all of the 
limitations of the independent claims, and much less the recitations of the dependent claims. 
These claims particularly point out and distinctly claim subject matter that appellants regard as 
their invention, which is clearly patentably distinguishable from the cited and applied references. 
A. Rejection of Independent Claim 1 Under 35 U.S.C. § 103(a) 

The Office Action rejected Claim 1 under 35 U.S.C. § 103(a) over Balogh in view of 
Cox. The Office Action states that Balogh teaches: 

(1) "an application program comprising [a] plurality of media clips in a database, with 
associated information describing each media clip, said media clips are in the form of images, as 
well as video clips and multimedia objects" (which the Office Action compares with "A method 
for searching a media clip database associated with a multimedia application program, wherein 
said media clip data base contains . . . that describes each associated media clip in said media clip 
data base, comprising:" as recited in Claim 1); 

(2) "a 'captioned which provides metadata in the form of a caption describing salient 
features of an image, bibliographic data, 'suggest fields 1 and attributes of said image, for each 
image" (which the Office Action compares with "keywords," as recited in Claim 1); 

(3) "a user performing an initial query . . . , resulting in retrieval of captions with images 
along with associated information" (which the Office Action compares with "(a) in response to a 
user selecting a media clip, retrieving information, including find similar clips indicia and 
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keywords, associated with said selected media clip from said media clip database," as recited in 
Claim 1); 

(4) "presenting the above captions, images, and information to a user for eventual query" 
(which the Office Action compares with "(b) simultaneously presenting to the user for selection 
by the user: (i) said keywords associated with said selected media clip;" as recited in Claim 1); 

(5) "image related criteria using human judgment via the use of a human 'captioner,' 
which verifies the quality of the image and information, and writes a caption, or description of 
the salient features/attributes of the images, as well as recording evoked emotional suggestions 
regarding said image" (which the Office Action compares with "based on human judgment 
regarding the content of the media clip," as recited in Claim 1); and, 

(6) "an additional query based upon the associated image description or bibliographic 
data of a previously selected image, the user can make further additions, deletions, and/or 
modifications to the associated information, if needed, prior to said additional query" (which the 
Office Action compares with "(c) in response to the user [creating] search criteria by selecting 
one or more of said keywords and/or find similar clips indicia associated with said selected 
media clip, retrieving all media clips in said media clip database that match the search criteria 
created by the user," as recited in Claim 1). 

The Office Action admits that Balogh fails to teach hidden criteria. The Office Action 
cites Cox as teaching "PicHunter, a media searching tool comprising a GUI interfaced with four 
displayed images (clips) from a clip database. A user selects an image, then optionally selects 
button 'GO 1 , after which said invention searches and presents a second set of images similar to 
what was initially selected." The Office Action states, "Cox's invention involves the analyzation 
of a user selected image, so that a similar set of images can be retrieved/presented. In choosing 
an image, the image attributes (i.e., color, shape, etc.) inherent within said image along with 
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various noted (hidden) features of each image (i.e., contrast, saturation, etc.), are used as a search 
criteria by the system," (which the Office Action compares with "including hidden criteria that 
identifies and/or groups media clips," as recited in Claim 1). 

The Office Action states: "It would have been obvious to one of ordinary skill in the art 
at the time of the invention to apply Cox to Balogh, providing Balogh the benefit of intelligent 
(accurate) analyzation of media images incorporating hidden search criteria, which may not be 
explicitly present in a user's vocabulary (or known to the user)." Appellants disagree. To 
establish a prima facie case of obviousness, there must be some suggestion or motivation to 
combine the reference teachings, there must be a reasonable expectation of success, and the prior 
art references must teach or suggest all the claim limitations. M.P.E.P. § 2143. 

Claim 1 recites: 

1. A method of searching a media clip database associated 
with a multimedia application program, wherein said media clip database 
contains information, including keywords and find similar clips indicia 
associated with media clips included in said media clip database, said find 
similar clips indicia including hidden criteria that identifies and/or groups 
media clips based on human judgment regarding the content of the media 
clip, said method comprising: 

(a) in response to a user selecting a media clip, retrieving 
information, including find similar clips indicia and keywords, associated 
with said selected media clip from said media clip database; 

(b) simultaneously presenting to the user for selection by 

the user: 

(i) said keywords associated with said selected media 

clip; and 

(ii) said find similar clips indicia associated with said 
selected media clip; and 

(c) in response to the user creating search criteria by 
selecting one or more of said keywords and/or said find similar clips 
indicia associated with said selected media clip, retrieving all media clips 
in said media clip database that match the search criteria created by the 
user. 
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1. There Is No Motivation to Combine the Cited References 

There is no motivation to combine Balogh and Cox to arrive at the claimed invention. 
Balogh is generally directed to a system and process for retrieving images using a natural 
language query and English language metadata associated with the image. In contrast, Cox is 
directed to a queryless multimedia database retrieval method and system that maintain a posterior 
probability distribution for use in selecting the next images to display to the user. There is no 
motivation for one of ordinary skill in the art to combine the teachings of a system for retrieving 
images using a language query with the teachings of a method of finding items based on a 
probability distribution over the items. 

2. The Cited References, Even if Combined for the Sake of Argument, Still Fail to 
Teach or Suggest All of the Claim Limitations 

The cited references, even if combined for the sake of argument, still fail to teach or 
suggest all of the limitations included in Claim 1. First, appellants disagree with the Office 
Action's discussion of the teachings of Balogh. Balogh fails to teach or suggest "in response to a 
user selecting a media clip, retrieving information, including find similar clips indicia and 
keywords, associated with said selected media clip from said media clip database." Referring to 
FIGURE 10, a flow diagram for a search engine process in accordance with the Balogh is shown. 
This flow diagram teaches a query arriving at an index service 1001, formulating a query 1012, 
matching the query against index data 1013, and returning matching images 1014. The query in 
Balogh begins with a user's input using standard keyword searching techniques. See, Col. 12, 
lines 9-11. Further, Col. 1 1, lines 53-67 state: 

For example, a user may type a query in ordinary English, such as 
"red trucks". Additionally, a user may enter query data into bibliographic 
fields, specifying, for example, images produced by a particular artist or 
agency. Finally, a user may also limit a query based on image 
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characteristics, such as predominant hue or image type, as described above 
in connection with FIG. 6. 

Thus, Balogh teaches returning matches based only on words or language entered in a query. In 

contrast, the present invention teaches beginning with a media clip (rather than words or 

language), and retrieving information "in response to a user selecting a media clip." 

Further, Balogh fails to teach or suggest "in response to the user creating search criteria 
by selecting one or more of said keywords and/or said find similar clips indicia associated with 
said selected media clip, retrieving all media clips in said media clip database that match the 
search criteria created by the user." While Balogh purportedly teaches that information inquiries 
are processed as in conventional keyword searching techniques (Col. 12, lines 9-11), and that a 
user may use information from the captions retrieved from a first search in a subsequent search 
(Col. 14, lines 40-59), Balogh does not disclose displaying keywords associated with a selected 
multimedia clip for selection by a user for subsequent searching. 

Finally, and most importantly, as the Office Action admits, Balogh does not teach or 
suggest a "find similar clips indicia including hidden criteria that identifies and/or groups media 
clips based on human judgment regarding the content of the media clip," such as, for example, 
artistic style criteria, color criteria, or shape criteria. At most, Balogh teaches that a "captioner" 
verifies the quality of the image data and the bibliographic data, and writes a short "caption," or a 
description of the salient features of the image, and to select certain attributes of the image. See, 
Col. 3, lines 21-29. Col. 3, lines 29-43 go on to state: 

The attributes may include, for example the type of image 
(photograph, computer-generated graphic, video clip, or other multimedia 
object, background pattern, portrait, abstract, aerial, or special effect), 
predominant hue, and image orientation (landscape or portrait). The 
captioner also provides as part of the metadata a suggest text field 
describing the emotional suggestions evoked by the image.... 
Collectively, the bibliographic data, the caption, the attributes and the 
suggests field are known as the 'metadata 1 associated with the image. 
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As the Office Action admits, the "caption" that identifies and/or groups media clips is not 
"hidden criteria." See also FIGURE 12; Col. 16, lines 40-62 ("match list command area 1212 
also permit the user to alter the display to provide additional information about the matches, such 
as bibliographical information"). 

Cox also fails to teach or suggest "a media clip database associated with a multimedia 
application program, wherein said media clip database contains information, including keywords 
and find similar clips indicia associated with media clips included in said media clip database, 
said find similar clips indicia including hidden criteria that identifies and/or groups media clips 
based on human judgment regarding the content of the media clip." Cox discloses a queryless 
multimedia database retrieval method and system that maintain a posterior probability 
distribution for use in selecting the next images to display to the user. See, Col. 2, lines 19-34. 
The selection system disclosed in Cox is entirely software driven and based upon probability 
distribution. No human judgment is involved in the criteria used to identify clips based on user 
selection. See, e.g., Col. 6, lines 1-22 (Table 1). 

In contrast, referring to FIGURE 2B, the hidden criteria described in the present 
invention identifies and/or groups media clips based on human judgment regarding the content 
of the media clips, such as artistic style 92, color or shape 94. The hidden criteria are stored in 
the clip catalogue when the clip is added to the catalogue. The hidden criteria are not changeable 
by a user. Further, the hidden criteria are not displayed to a user. Only the broad indicia— artistic 
style, color, or shape— are displayed. See, e.g., Application, page 10, lines 26-31. 
B. Rejection of Dependent Claims 2-15 Under 35 U.S.C. $ 103(a) 

With respect to Claims 2-15, all of which depend directly or indirectly from Claim 1, it is 
clear that the subject matter of these claims is also not taught or suggested by the cited and 
applied references, namely, Balogh or Cox. Claims 2-15 all have limitations that are clearly not 
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taught or suggested by any of the cited and applied references, particularly when the limitations 
are considered in combination with these recitations of the claims from which these claims 
individually depend. Thus, Claims 2-15 are submitted to be allowable for reasons in addition to 
the reasons why Claim 1 is submitted to be allowable. 
C. Rejection of Independent Claim 16 Under 35 U.S.C. § 103(a) 

The Office Action rejected Claim 16 under 35 U.S.C. § 103(a) over Balogh in view of 
Cox. The Office Action states Balogh teaches: (1) "an application program comprising [a] 
plurality of media clips in a database, with associated information describing each media clip, 
said media clips are in the form of images, as well as video clips and multimedia objects" and 
"associated data regarding what a particular media images suggests, which is indicative of a 
visual thesaurus" (which the Office Action appears to compare with "A method for providing a 
user interface for a visual thesaurus for a media clip database associated with a multimedia 
application program, wherein said media clip database contains information, including find 
similar clips indicia and keywords associated with media clips included in said media clip 
database, said find similar clips indicia including hidden criteria that identifies and/or groups 
media clips based on human judgment regarding the content of the media clip, said method 
comprising:" as recited in Claim 16); 

(2) "a 'captioned which provides metadata in the form of a caption describing salient 
features of an image, bibliographic data, Suggest fields 1 and attributes of said image, for each 
image" (which the Office Action compares with "keywords," as recited in Claim 16); 

(3) "a user performing an initial query . . . , resulting in retrieval of captions with images 
along with associated information presenting said captions, images and information to a user" 
(which the Office Action compares with "directly in response to a user selecting a media clip 
from said media clip database," as recited in Claim 16); 

LAW OFFICES OF 
CHRISTENSEN O'CONNOR JOHNSON KINDNESS*^ 
1420 Fifth Avenue 
Suite 2800 
Seattle, Washington 98101 
-21- 206.682.8100 

MSFni2767199.DOC 



(4) "an additional query based upon the associated image description or bibliographic 
data of a previously selected image, the user can make further additions, deletions, and/or 
modifications to the associated information, if needed, prior to said additional query" (which the 
Office Action appears to compare with "displaying to the user an option for finding similar media 
clips that have an associated find similar clips indicia and/or a keyword that matches the find 
similar clips indicia, human judgment and/or a keyword associated with the selected clip," as 
recited in Claim 16); and 

(5) "image related criteria using human judgment via the use of a human 'captioner,' 
which verifies the quality of the image and information, and writes a caption, or description of 
the salient features/attributes of the images, as well as recording evoked emotional suggestions 
regarding said image" (which the Office Action compares with "based on human judgment 
regarding the content of the media clip," as recited in Claim 16). 

The Office Action admits that Balogh fails to teach hidden criteria. The Office Action 
cites Cox as teaching "PicHunter, a media searching tool comprising a GUI interface with four 
displayed images (clips) from a clip database. A user selects an image, then optionally selects 
button 'GO', after which said invention searches and presents a second set of images similar to 
what was initially selected." The Office Action states, "Cox's invention involves the analyzation 
of a user selected image, so that a similar set of images can be retrieved/presented" (which the 
Office Action compares with "find similar clips indicia including hidden criteria . . . ," as recited 
in Claim 16). 

The Office Action states that it would have been obvious to one of ordinary skill in the art 
at the time of the invention to apply Cox to Balogh, providing Balogh the benefit of intelligent 
(accurate) analyzation of media images incorporating hidden search criteria, which may not be 
explicitly present in a user's vocabulary (or known to the user). Appellants disagree. 
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Claim 16 recites: 

16. A method for providing a user interface for a visual 
thesaurus for a media clip database associated with a multimedia 
application program, wherein said media clip database contains 
information, including find similar clips indicia and keywords associated 
with media clips included in said media clip database, said find similar 
clips indicia including hidden criteria that identifies and/or groups media 
clips based on human judgment regarding the content of the media clip, 
said method comprising: 

directly in response to a user selecting a media clip from said 
media clip database, displaying to the user an option for finding similar 
media clips that have an associated find similar clips indicia and/or a 
keyword that matches the find similar clips indicia, human judgment 
and/or a keyword associated with the selected clip. 

1. There Is No Motivation to Combine the Cited References 

As discussed above in Part VIII. A. 1, there is no motivation to combine Balogh and Cox 
to arrive at the claimed invention. 

2. The Cited References, Even if Combined for the Sake of Argument, Still Fail to 
Teach or Suggest All of the Claim Limitations 

The cited references, even if combined for the sake of argument, still fail to teach or 
suggest all of the limitations included in Claim 16. First, appellants disagree with the Office 
Action's discussion of the teachings of Balogh. Balogh fails to teach or suggest "directly in 
response to a user selecting a media clip from said media clip database, displaying to the user an 
option for finding similar media clips that have an associated find similar clips indicia and/or a 
keyword that matches the find similar clips indicia, human judgment and/or a keyword associated 
with the selected clip." Referring to FIGURE 10, a flow diagram for a search engine process in 
accordance with the Balogh reference is shown. This flow diagram teaches a query arriving at an 
index service 1001, formulating a query 1012, matching the query against index data 1013, and 
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returning matching images 1014. The query in Balogh begins with a user's input using standard 
keyword searching techniques. See, Col. 12, lines 9-11. Further, Col. 11, lines 53-67 state: 

For example, a user may type a query in ordinary English, such as 
"red trucks". Additionally, a user may enter query data into bibliographic 
fields, specifying, for example, images produced by a particular artist or 
agency. Finally, a user may also limit a query based on image 
characteristics, such as predominant hue or image type, as described above 
in connection with FIG. 6. 

Thus, Balogh teaches returning matches based only on words or language entered in a query. In 

contrast, the present invention teaches beginning with a media clip (rather than words or 

language), and retrieving information "directly in response to a user selecting a media clip." 

Balogh purportedly teaches that information inquiries are processed as in conventional 
keyword searching techniques (Col. 12, lines 9-11), and that a user may use information from the 
captions retrieved from a first search in a subsequent search (Col. 14, lines 40-59). However, 
Balogh does not disclose displaying to the user an option (for selection) for finding similar clips, 
without entering a new query with new search language. 

In addition, and more importantly, as described above in Part VIII.A.2, the Office Action 
admits that Balogh does not teach or suggest "find similar clips indicia including hidden criteria 
that identifies and/or groups media clips based on human judgment regarding the content of the 
media clip," such as, for example, artistic style criteria, color criteria, or shape criteria. As fully 
discussed in Part VIII.A.2, this Balogh deficiency is not cured by the teaching of Cox. 
D. Rejection of Dependent Claims 17-22 Under 35 U.S.C. § 103(a) 

With respect to Claims 17-22, all of which depend directly or indirectly from Claim 16, it 
is clear that the subject matter of these claims is also not taught or suggested by the cited and 
applied references, namely, Balogh or Cox. Claims 17-22 all have limitations that are clearly not 
taught or suggested by any of the cited and applied references, particularly when the limitations 
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are considered in combination with these recitations of the claims from which these claims 
individually depend. Thus, Claims 17-22 are submitted to be allowable for reasons in addition to 
the reasons why Claim 16 is submitted to be allowable. 
E. Rejection of Independent Claim 24 Under 35 U.S.C. § 103(a) 

The Office Action rejected Claim 24 under 35 U.S.C. § 103(a) over Balogh in view of 
Cox. The Office Action states that Balogh teaches: 

(1) "data entry, disambiguation, and database processors within a SUN 
SPARCSTATION" (which the Office Action compares with "a processing unit," as recited in 
Claim 24); 

(2) "a computer readable medium (i.e., diskette or hard drive) used for holding 
instructions and utilized within a computer" (which the Office Action appears to compare with "a 
storage medium coupled to the processing unit, the storage medium storing program code 
implemented by the processing unit," as recited in Claim 24); 

(3) "an application program comprising [a] plurality of media clips in a database, with 
associated information describing each media clip, said media clips are in the form of images, as 
well as video clips and multimedia objects, said clips subject to user query and selection" (which 
the Office Action compares with "providing an interface for a user to select a media clip ... in 
said media clip database," as recited in Claim 24); 

(4) "a 'captioned which provides metadata in the form of a caption describing salient 
features of an image, bibliographic data, Suggest fields' and attributes of said image, for each 
image" (which the Office Action compares with "keywords," as recited in Claim 24); 

(5) "image related criteria using human judgment via the use of a human ! captioner, f 
which verifies the quality of the image and information, and writes a caption, or description of 
the salient features/attributes of the images, as well as recording evoked emotional suggestions 
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regarding said image" (which the Office Action compares with "based on human judgment 
regarding the content of the media clip," as recited in Claim 24); and 

(6) "a user performing an initial query . . . , resulting in retrieval of captions with images 
along with associated information, as well as an additional query based upon the associated 
image description or bibliographic data of a previously selected image, the user can make further 
additions, deletions, and/or modifications to the associated information, if needed, prior to said 
additional query" (which the Office Action appears to compare with "(ii) providing an interface 
for the user to select search criteria based on find similar clips indicia and/or a keyword 
associated with said selected media clip; and (iii) in response to the user selecting the media clip 
and the search criteria, retrieving all media clips in the media clip database that have associated 
find similar clips indicia and/or a keyword that matches the selected search criteria for the 
selected media clip," as recited in Claim 24). 

The Office Action admits that Balogh fails to teach hidden criteria. The Office Action 
cites Cox as teaching "PicHunter, a media searching tool comprising a GUI interface with four 
displayed images (clips) from a clip database. A user selects an image, then optionally selects 
button 'GO', after which said invention searches and presents a second set of images similar to 
what was initially selected." The Office Action states, "Cox's invention involves the analyzation 
of a user selected image, so that a similar set of images can be retrieved/presented" (which the 
Office Action compares with "find similar clips indicia including hidden criteria that identifies 
and/or groups media clips," as recited in Claim 24). 

The Office Action states that it would have been obvious for one of ordinary skill in the 
art at the time of the invention to apply Cox to Balogh, providing Balogh the benefit of 
intelligent (accurate) analyzation of media images incorporating hidden search criteria, which 
may not be explicitly present in a user's vocabulary (or known to the user). Appellants disagree. 
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Claims 24 recites: 

24. An apparatus for searching a plurality of media clips, 
comprising: 

(a) a processing unit; and 

(b) a storage medium coupled to the processing unit, the 
storage medium storing program code implemented by the processing unit 
for: 

(i) providing an interface for a user to select a media 
clip from a media clip database associated with a multimedia application 
program, wherein said media clip database contains information, including 
find similar clips indicia and keywords associated with media clips in said 
media clip database, said find similar clips indicia including hidden 
criteria that identifies and/or groups media clips based on human judgment 
regarding the content of the media clip, said apparatus; 

(ii) providing an interface for the user to select search 
criteria based on find similar clips indicia and/or a keyword associated 
with said selected media clip; and 

(iii) in response to the user selecting the media clip and 
the search criteria, retrieving all media clips in the media clip database that 
have associated find similar clips indicia and/or a keyword that matches 
the selected search criteria for the selected media clip. 

1. There Is No Motivation to Combine the Cited References 

As discussed above in Part VIII.A.l, there is no motivation to combine Balogh and Cox 
to arrive at the claimed invention. 

2. The Cited References, Even if Combined for the Sake of Argument, Still Fail to 
Teach or Suggest All of the Claim Limitations 

The cited references, even if combined for the sake of argument, still fail to teach or 
suggest all of the claim limitations in Claim 24. First, appellants disagree with the Office 
Action's discussion of the teachings of Balogh. Balogh fails to teach or suggest "providing an 
interface for a user to select a media clip from a media clip database associated with a 
multimedia application program." Referring to FIGURE 10, a flow diagram for a search engine 



LAW OFFICES OF 
CHRISTENSEN O'CONNOR JOHNSON KINDNESS 1 
1420 Fifth Avenue 
Suite 2800 
Seattle, Washington 98101 
-27- 206.682.8100 

MSFR12767199.DOC 



process in accordance with the Balogh is shown. This flow diagram teaches a query arriving at 
an index service 1001, formulating a query 1012, matching the query against index data 1013, 
and returning matching images 1014. The query in Balogh begins with a user's input using 
standard keyword searching techniques. See, Col. 12, lines 9-11. Further, Col. 11, lines 53-67 
state: 

For example, a user may type a query in ordinary English, such as 
"red trucks". Additionally, a user may enter query data into bibliographic 
fields, specifying, for example, images produced by a particular artist or 
agency. Finally, a user may also limit a query based on image 
characteristics, such as predominant hue or image type, as described above 
in connection with FIG. 6. 

Thus, Balogh teaches returning matches based only on words or language entered in a query. In 

contrast, the present invention teaches beginning with a media clip (rather than words or 

language), and retrieving information "in response to a user selecting a media clip." 

Further, Balogh fails to teach or suggest "providing an interface for the user to select 
search criteria based on find similar clips indicia and/or a keyword associated with said selected 
media clip." While Balogh purportedly teaches that information inquiries are processed as in 
conventional keyword searching techniques (Col. 12, lines 9-11), and that a user may use 
information from the captions retrieved from a first search in a subsequent search (Col. 14, 
lines 40-59), Balogh does not disclose keywords associated with a selected media clip for 
selection by a user for subsequent searching. 

Finally, and more importantly, as described above in Part VIII.A.2, the Office Action 
admits that Balogh does not teach or suggest "find similar clips indicia including hidden criteria 
that identifies and/or groups media clips based on human judgment regarding the content of the 
media clip," such as, for example, artistic style criteria, color criteria, or shape criteria. As fully 
discussed in Part VIII.A.2., this Balogh deficiency is not cured by the teaching of Cox. 
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F. Rejection of Dependent Claims 25-32, 34-35, and 37 Under 35 U.S.C. $ 103(a) 

With respect to Claims 25-32, 34-35, and 37, all of which depend directly or indirectly 
from Claim 24, it is clear that the subject matter of these claims is also not taught or suggested by 
the cited and applied references, namely, Balogh or Cox. Claims 25-32, 34-35, and 37 all have 
limitations that are clearly not taught or suggested by any of the cited and applied references, 
particularly when the limitations are considered in combination with these recitations of the 
claims from which these claims individually depend. Thus, Claims 25-32, 34-35, and 37 are 
submitted to be allowable for reasons in addition to the reasons why Claim 24 is submitted to be 
allowable. 

G. Rejection of Independent Claim 38 Under 35 U.S.C. S 103(a) 

The Office Action rejected Claim 38 under 35 U.S.C. § 103(a) over Balogh in view of 
Cox. The Office Action states that Balogh teaches: 

(1) "an application program comprising [a] plurality of media clips in a database, with 
associated information describing each media clip, said media clips are in the form of images, as 
well as video clips and multimedia objects" (which the Office Action appears to compare with 
"An apparatus for providing a user interface for a visual thesaurus for a media clip database 
associated with a multimedia application program, wherein the media clip database contains 
information, . . . associated with media clips in said media clip database," as recited in Claim 38); 

(2) "a 'captioned which provides metadata in the form of a caption describing salient 
features of an image, bibliographic data, 'suggest fields' and attributes of said image, for each 
image" (which the Office Action compares with "keywords," as recited in Claim 38); 

(3) a "data entry, disambiguation, and database processors within a SUN 
SPARCSTATION" (which the Office Action compares with "a processing unit," as recited in 
Claim 38); 
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(4) "a computer readable medium (i.e., diskette or hard drive) used for holding 
instructions and utilized within a computer["?] (which the Office Action appears to compare 
with "a storage medium coupled to the processing unit, the storage medium storing program code 
implemented by the processing unit," as recited in Claim 38); 

(5) "a user performing an initial query . . . , resulting in retrieval of captions with images 
along with associated information presenting said captions, images and information to a user, as 
well as an additional query based upon the associated image description or bibliographic data of 
a previously selected image, the user can make further additions, deletions, and/or modifications 
to the associated information, if needed, prior to said additional query" (which the Office Action 
appears to compare with "displaying to a user an option for finding similar media clips that have 
associated find similar clips indicia hidden criteria, and/or a keyword that matches an associated 
keyword for a selected media clip, directly in response to the user selecting the media clip," as 
recited in Claim 38); and 

(6) "image related criteria using human judgment via the use of a human 'captioner,' 
which verifies the quality of the image and information, and writes a caption, or description of 
the salient features/attributes of the images, as well as recording evoked emotional suggestions 
regarding said image" (which the Office Action compares with "based on human judgment 
regarding the content of the media clip," as recited in Claim 38). 

The Office Action admits that Balogh fails to teach hidden criteria. The Office Action 
cites Cox as teaching "PicHunter, a media searching tool comprising a GUI interface with four 
displayed images (clips) from a clip database. A user selects an image, then optionally selects 
button 'GO 1 , after which said invention searches and presents a second set of images similar to 
what was initially selected." The Office Action states, "Cox's invention involves the analyzation 
of a user selected image, so that a similar set of images can be retrieved/presented" (which the 
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Office Action compares with "find similar clips indicia including hidden criteria . . . as recited 
in Claim 38). 

The Office Action states that it would have been obvious for one of ordinary skill in the 
art at the time of the invention to apply Cox to Balogh, providing Balogh the benefit of 
intelligent (accurate) analyzation of media images incorporating hidden search criteria, which 
may not be explicitly present in a user's vocabulary (or known to the user). Appellants disagree. 

Claim 38 recites: 

38. An apparatus for providing a user interface for a visual 
thesaurus for a media clip database associated with a multimedia 
application program, wherein the media clip database contains 
information, including find similar clips indicia and keywords associated 
with media clips in said media clip database, said find similar clips indicia 
including hidden criteria that identifies and/or groups media clips based on 
human judgment regarding the content of the media clip, said apparatus 
comprising: 

(a) a processing unit; and 

(b) a storage medium coupled to the processing unit, the 
storage medium storing program code implemented by the processing unit 
for displaying to a user an option for finding similar media clips that have 
associated find similar clips indicia hidden criteria, and/or a keyword that 
matches an associated keyword for a selected media clip, directly in 
response to the user selecting the media clip. 

1. There Is No Motivation to Combine the Cited References 

As discussed above in Part VIII. A. 1 , there is no motivation to combine Balogh and Cox 
to arrive at the claimed invention. 

2. The Cited References, Even if Combined for the Sake of Argument, Still Fail to 
Teach or Suggest All of the Claim Limitations 

The cited references, even if combined for the sake of argument, still fail to teach or 
suggest all of the claim limitations in Claim 38. First, appellants disagree with the Office 
Action's discussion of the teachings of Balogh. Balogh fails to teach or suggest "displaying to a 
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user an option for finding similar media clips that have associated find similar clips indicia 
hidden criteria, and/or a keyword that matches an associated keyword for a selected media clip, 
directly in response to the user selecting the media clip'' Referring to FIGURE 10, a flow 
diagram for a search engine process in accordance with the Balogh reference is shown. This flow 
diagram teaches a query arriving at an index service 1001, formulating a query 1012, matching 
the query against index data 1013, and returning matching images 1014. The query in Balogh 
begins with a user's input using standard keyword searching techniques. See, Col. 12, lines 9-11. 
Further, Col. 11, lines 53-67 state: 

For example, a user may type a query in ordinary English, such as 
"red trucks". Additionally, a user may enter query data into bibliographic 
fields, specifying, for example, images produced by a particular artist or 
agency. Finally, a user may also limit a query based on image 
characteristics, such as predominant hue or image type, as described above 
in connection with FIG. 6. 

Thus, Balogh teaches returning matches based only on words or language entered in a query. In 

contrast, the present invention teaches beginning with a media clip (rather than words or 

language), and retrieving information "in response to a user selecting a media clip." 

Balogh purportedly teaches that information inquiries are processed as in conventional 
keyword searching techniques (Col. 12, lines 9-11), and that a user may use information from the 
captions retrieved from a first search in a subsequent search (Col. 14, lines 40-59). However, 
Balogh does not disclose displaying to a user an option (for selection) for finding similar media 
clips, without entering a new query with new search language. 

In addition, and more importantly, as described above in Part VIII.A.2, the Office Action 
admits that Balogh does not teach or suggest a "find similar clips indicia including hidden criteria 
that identifies and/or groups media clips based on human judgment regarding the content of the 
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media clip," such as, for example, artistic style criteria, color criteria, or shape criteria. As fully 
discussed in Part VIII.A.2, this Balogh deficiency is not cured by the teaching of Cox. 
H. Rejection of Claims 39-44 Under 35 U.S.C. § 103(a) 

With respect to Claims 39-44, all of which depend directly or indirectly from Claim 38, it 
is clear that the subject matter of these claims is also not taught or suggested by the cited and 
applied references, namely, Balogh or Cox. Claims 39-44 all have limitations that are clearly not 
taught or suggested by any of the cited and applied references, particularly when the limitations 
are considered in combination with these recitations of the claims from which these claims 
individually depend. Thus, Claims 39-44 are submitted to be allowable for reasons in addition to 
the reasons why Claim 38 is submitted to be allowable. 

EX. CONCLUSION 

In view of the foregoing remarks, appellants submit that all of the claims in the present 
application are clearly patentably distinguishable over the teachings of Balogh et al. and Cox 
et al. In addition to the other differences, as discussed above, neither reference teaches or even 
remotely suggests a "find similar clips indicia including hidden criteria that identifies and/or 
groups media clips based on human judgment regarding the content of the media clip," a 
recitation included in all of the independent claims on Appeal. Therefore, it is submitted that the 
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rejections of Claims 1-9, 11, 12, 14-32, 34, 35, and 37-44 were erroneous, and reversal of this 
decision is respectfully requested. 

Respectfully submitted, 

CHRISTENSEN O'CONNOR 
JOHNSON fCINDNESSPLLC 




jary S. Kindness 
^Registrati/n No. 22, 1 78 
Direct Dial No. 206.695.1702 

I hereby certify that this correspondence is being deposited with the U.S. Postal Service in a sealed 
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Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313-1450, on the below date. 
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X. 



APPENDIX OF CLAIMS INVOLVED IN THE APPEAL 



1. A method of searching a media clip database associated with a multimedia 
application program, wherein said media clip database contains information, including keywords 
and find similar clips indicia associated with media clips included in said media clip database, 
said find similar clips indicia including hidden criteria that identifies and/or groups media clips 
based on human judgment regarding the content of the media clip, said method comprising: 

(a) in response to a user selecting a media clip, retrieving information, 
including find similar clips indicia and keywords, associated with said selected media clip from 
said media clip database; 

(b) simultaneously presenting to the user for selection by the user: 

(i) said keywords associated with said selected media clip; and 

(ii) said find similar clips indicia associated with said selected media 

clip; and 

(c) in response to the user creating search criteria by selecting one or more of 
said keywords and/or said find similar clips indicia associated with said selected media clip, 
retrieving all media clips in said media clip database that match the search criteria created by the 
user. 

2. The method of Claim 1, further comprising presenting a user with all of the 
retrieved clips. 

3. The method of Claim 2, further comprising, in response to a user selecting one of 
the retrieved clips and the user creating a search criteria for the newly selected clip, retrieving all 
clips in the clip database that match the search criteria for the newly selected media clip. 

4. The method of Claim 1, further comprising presenting a plurality of media clips 
available for selection to a user. 
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5. The method of Claim 4, wherein the media clips are based on a media type. 

6. The method of Claim 1, wherein said human judgment is based on artistic style. 

7. The method of Claim 1, wherein said human judgment is based on color. 

8. The method of Claim 1, wherein said human judgment is based on shape. 

9. The method of Claim 1, wherein said human judgment is based on both color and 

shape. 

1 1 . The method of Claim 1 , wherein the keywords are a list of keywords. 

12. The method of Claim 1, wherein the keywords designate the format of the clip. 

14. The method of Claim 1 5 wherein said retrieving all clips in said clip database that 
match the search criteria created by the user applies only to clips having a media type that 
matches the media type of the selected media clip. 

15. A computer-readable medium having computer-executable instructions for 
performing the method recited in any one of Claims 1-9, 11, 12, or 14. 

16. A method for providing a user interface for a visual thesaurus for a media clip 
database associated with a multimedia application program, wherein said media clip database 
contains information, including find similar clips indicia and keywords associated with media 
clips included in said media clip database, said find similar clips indicia including hidden criteria 
that identifies and/or groups media clips based on human judgment regarding the content of the 
media clip, said method comprising: 

directly in response to a user selecting a media clip from said media clip database, 
displaying to the user an option for finding similar media clips that have an associated find 
similar clips indicia and/or a keyword that matches the find similar clips indicia, human 
judgment and/or a keyword associated with the selected clip. 

17. The method of Claim 16, wherein the user interface comprises: 
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(a) a graphical indication of the selected clip; and 

(b) a button for the option for finding similar clips to the selected clip. 

18. The method of Claim 16, further comprising, in response to a user selecting the 
option for finding similar clips, displaying means for the user to select search criteria for finding 
similar clips, wherein said search criteria includes a keyword that matches the find similar clips 
indicia, human judgment and/or a keyword associated with said selected clip. 

19. The method of Claim 18, wherein means for the user to select search criteria for 
finding similar clips is a fly-out window. 

20. The method of Claim 16, further comprising displaying to the user an option for 
inserting the selected clip into a document. 

21. The method of Claim 16, further comprising displaying to the user an option for 
previewing the selected clip. 

22. The method of Claim 16, further comprising displaying to the user an option to 
add the clip to one or more categories. 

23. A computer-readable medium having computer-executable instructions for 
performing the method recited in any one of Claims 16-22. 

24. An apparatus for searching a plurality of media clips, comprising: 

(a) a processing unit; and 

(b) a storage medium coupled to the processing unit, the storage medium 
storing program code implemented by the processing unit for: 

(i) providing an interface for a user to select a media clip from a 
media clip database associated with a multimedia application program, wherein said media clip 
database contains information, including find similar clips indicia and keywords associated with 
media clips in said media clip database, said find similar clips indicia including hidden criteria 
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that identifies and/or groups media clips based on human judgment regarding the content of the 
media clip, said apparatus; 

(ii) providing an interface for the user to select search criteria based on 
find similar clips indicia and/or a keyword associated with said selected media clip; and 

(iii) in response to the user selecting the media clip and the search 
criteria, retrieving all media clips in the media clip database that have associated find similar 
clips indicia and/or a keyword that matches the selected search criteria for the selected media 
clip. 

25. The apparatus of Claim 24, wherein the program code further presents a user with 
all of the retrieved clips. 

26. The apparatus of Claim 25, wherein the program code further retrieves all clips in 
the clip database that match a search criteria for a newly selected media clip, in response to a user 
selecting one of the retrieved clips and the user selecting the search criteria for the newly selected 
clip. 

27. The apparatus of Claim 24, wherein the program code further presents a plurality 
of media clips available for selection to a user. 

28. The apparatus of Claim 27, wherein the media clips are based on a media type. 

29. The apparatus of Claim 24, wherein said human judgment is based on artistic 

style. 

30. The apparatus of Claim 24, wherein said human judgment is based on color. 

3 1 . The apparatus of Claim 24, wherein said human judgment is based on shape. 

32. The apparatus of Claim 24, wherein said human judgment is based on both color 
and shape. 

34. The apparatus of Claim 24, wherein the keywords are a list of keywords. 
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35. The apparatus of Claim 24, wherein the keywords designate the format of the clip. 

37. The apparatus of Claim 24, wherein said retrieving all clips in said clip database 
that match the search criteria for the selected media clip applies only to clips having a media type 
that matches the media type of the selected media clip. 

38. An apparatus for providing a user interface for a visual thesaurus for a media clip 
database associated with a multimedia application program, wherein the media clip database 
contains information, including find similar clips indicia and keywords associated with media 
clips in said media clip database, said find similar clips indicia including hidden criteria that 
identifies and/or groups media clips based on human judgment regarding the content of the 
media clip, said apparatus comprising: 

(a) a processing unit; and 

(b) a storage medium coupled to the processing unit, the storage medium 
storing program code implemented by the processing unit for displaying to a user an option for 
finding similar media clips that have associated find similar clips indicia hidden criteria, and/or a 
keyword that matches an associated keyword for a selected media clip, directly in response to the 
user selecting the media clip. 

39. The apparatus of Claim 38, wherein the user interface comprises: 

(a) a graphical indication of the selected clip; and 

(b) a button for the option for finding similar clips to the selected clip. 

40. The apparatus of Claim 38, wherein the program code displays means for a user to 
select search criteria that is based on find similar clips indicia and/or a keyword associated with 
said selected clip for finding similar clips, in response to the user selecting the option for finding 
similar clips. 
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41. The apparatus of Claim 40, wherein means for the user to select search criteria for 
finding similar clips is a fly-out window. 

42. The apparatus of Claim 38 wherein the program code further displays to the user 
an option for inserting the selected clip into a document. 

43. The apparatus of Claim 38 wherein the program code further displays to the user 
an option for previewing the selected clip. 

44. The apparatus of Claim 38 wherein the program code further displays to the user 
an option to add the clip to one or more categories. 
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